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CHAPTER  1 
INTRODUCTION 

1.1  CONCEPT  OF  DECISION  MAKING  IN  MANAGEMENT 

Recently  there  have  been  a  lot  of  changes  in  business  organizations 
and  they  have  become  enormously  complex  undertakings.  The  increasing 
business  strategies  and  approaches  as  well  as  scarce  resources  have 
made  them  even  more  complex.  During  the  past  couple  of  decades  the 
tendency  has  been  for  blending  mathematics  with  business  administration 
resulting  in  more  efficient  organization.  Similarly  the  inter- 
mediary  fields,    such  as  industrial  engineering  and  operations 
research  have  been  evolved  and  developed. 

Industrial  engineering  is  concerned  with  the  design,  improvement 
and  installation  of  integrated  systems  of  men,  materials  and  equipment. 
It  draws  upon  specialized  knowledge  and  skill  in  the  mathematical, 
physical  and  social  sciences,  together  with  the  principles  and  methods 
of  engineering  analysis  and  design,  to  specify,  to  predict  and  to 
evaluate  the  results  to  be  obtained  from  such  systems. 

With  the  scarce  resources  man  has  recognized  their  importance  and 
has  been  constantly  trying  to  emphasize  their  best  use.  The  resources 
can  be  of  any  kind,  such  as  material,  energy,  manpower,  etc. 

In  the  span  of  a  day  the  administrator  has  to  make  a  number  of  decisions. 
The  selection  of  one  action  or  sequence  of  actions  from  a  number  of 
alternative  possible  actions  is  known  as  a  decision.   Decision  making 
in  business,  in  the  past,  was  practiced  more  as  an  art  than  a  science. 
Increasing  costs  are  associated  with  decision  making.   Businesses  are 
larger;  each  decision  involves  a  larger  outlay  of  time,  money  and  resources. 


Some  problems  in  decision  making  can  be  quantified.  Objective 
solutions  optimizing  particular  goals  can  be  obtained  through  the 
proper  application  of  techniques  for  the  analysis  of  quantitative  data. 
Some  of  these  techniques  are  relatively  simple  and  nonmathematical, 
others  involve  high  levels  of  mathematical  proficiency.  With  the 
development  of  electronic  data  processing  equipment  these  techniques 
are  brought  to  the  access  of  many  potential  users. 

The  following  list  is  suggested  here  because  it  is  short  and  well 
suited  to  the  use  of  quantitative  information  as  the  basis  for  decision 
making  (7). 

i.  Define  the  problem, 
ii.   Determine  the  assumptions  and /or  limitations  which  affect 
the  solution, 
iii.  Identify  the  possible  courses  of  action, 
iv.  Isolate  the  decision  making  criteria. 
v.   Determine  and  compare  the  possible  outcomes  and  the 
probability  of  success  in  reaching  the  objective  for 
the  various  courses  of  action, 
vi.  Make  the  decision  (select  a  course  of  action), 
vii.   Implement  the  decision, 
viii.  Monitor  the  results  of  the  decision. 

1.2  STATE  INCREMENT  DYNAMIC  PROGRAMMING 

It  was  realized  during  the  period  following  World  War  II  that 
there  were  a  large  number  of  activities  which  could  be  classified  as 
multistage  decision  processes  (5) .   It  was  also  found  that  there  was 
a  shortage  in  available  techniques  to  solve  these  problems  and  those 


which  existed  were  not  versatile  enough.  Recognition  of  these  facts 

led  to  the  evolution  of  many  new  techniques  and  one  of  them  was  dynamic 

programming  (3) .   This  was  a  new  approach  based  on  the  use  of  the 

functional  equation  and  the  principle  of  optimality.  The  principle 

of  optimality  can  be  stated  as, 

"an  optimal  policy  has  a  property  that  whatever  the  initial 
state  and  initial  decisions  are,  the  remaining  decisions  must 
constitute  an  optimal  policy  with  regard  to  the  state  result- 
ing from  the  first  decision." 

Initially  the  emphasis  was  on  those  processes  which  were  specifically 
posed  as  multistage  decision  processes.   Then  all  the  processes  to  which 
dynamic  programming  could  be  applied  were  considered.   In  1955  R.  Bellman 
and  others  (7)  began  a  systematic  study  of  the  computational  feasibility 
of  dynamic  programming.   They  collected  a  number  of  optimization  problems 
from  many  different  fields  and  applied  their  methods  in  many  different 
ways. 

Despite  the  attractive  features  of  the  standard  algorithm  (dynamic 
programming),  its  applicability  thus  far  has  been  limited  to  relatively 
simple  cases.   This  is  due  to  the  large  computational  requirements  of 
this  algorithm.   The  most  severe  restriction  is  generally  the  high  amount 
of  fast  storage  memory  required  to  implement  the  basic  calculations. 
Another  difficulty  is  the  amount  of  computing  time  required  to  obtain 
the  complete  solution.   Thus,  while  dynamic  programming  is  frequently 
used  as  an  analytical  and  conceptual  tool,  the  computational  difficulties 
associated  with  the  standard  algorithm  have  severely  limited  its  appli- 
cation to  large  scale  optimization  problems. 

Along  with  other  methods  to  overcome  the  difficulty  of  fast  storage 
memory  requirement  in  dynamic  programming,  state  increment  dynamic 


programming  (16,17,18)  is  a  good  substitute.  R.  E.  Larson  was  the 
first  to  show  the  application  of  this  new  method  to  the  problems  which 
could  be  solved  by  dynamic  programming  but  are  likely  to  face  the 
'dimensionality  difficulty.'   This  method  also  is  based  on  Bellman's 
'principle  of  optimality. '  This  procedure  retains  the  desirable 
properties  of  the  standard  algorithm  but  has  a  reduced  computational 
requirements.  This  procedure  always  reduces  high-speed  storage  memory 
requirement,  often  by  orders  of  magnitude.   In  a  number  of  cases  a 
substantial  reduction  in  computing  time  can  be  achieved  as  well.   Thus 
state  increment  dynamic  programming  represents  a  significant  step  in 
increasing  the  range  of  optimization  problems  that  can  be  solved  with 
state-of-the-art  computer  facilities. 

1.3  PURPOSE  OF  THIS  STUDY 

Industrial  engineers  deal  with  many  different  varieties  of  engineer- 
ing as  well  as  business  problems  related  to  engineering.  Decision  making 
problems  are  one  of  the  varieties  which  an  industrial  engineer  should 
know  'how  to  tackle.'   There  are  many  ways  or  techniques  to  solve  these 
kinds  of  problems  but  an  engineer  always  looks  for  more  efficient 
ways. 

The  purpose  of  this  study  is  to  evaluate  the  effectiveness  of  state 
increment  dynamic  programming  in  solving  industrial  management  problems 
which  involve  nonlinear  difference  equations. 

Other  computational  procedures,  such  as  gradient  technique,  second 
variation  method,  and  invariant  imbedding,  etc.,  could  also  be  used  for 
solving  the  problems  of  this  nature,  but  they  will  not  be  discussed  here 
since  they  are  outside  the  scope  of  this  study. 
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1.4  HYDRO-DYNAMIC  JOURNAL  BEARING  DESIGN 

With  the  invention  of  data  processing  equipment,  many  fields  of 
science  have  changed  their  manner  of  problem  solution.   At  the  same 
time  mechanical  engineers  concerned  with  designs  have  also  changed 
their  approach  to  the  design  problems. 

Mechanical  designs  used  to  be  based  on  the  standard  design  equa- 
tions and  were  designed  on  the  idea  of  'safe'  design.  Now,  optimization 
techniques  have  improved  their  approach  and  helped  them  to  carry  out 
'safe*  and  'economical'  designs. 

Chapter  2  of  this  study  is  concerned  with  a  basic  mechanical  design, 
hydrodynamic  journal  bearings.  Master's  research  work  was  started  with 
this  work,  and  because  of  insufficient  information  we  were  not  able  to 
extend  the  problem  and  the  work  was  discontinued. 


CHAPTER  2 
OPTIMIZATION  OF  HYDRODYNAMIC  JOURNAL  BEARING  DESIGN 

2.1  GENERAL 

In  these  days  of  automization,  automated  mechanical  designs  have 
become  a  fertile  field  for  mechanical  engineers.   For  many  years  they 
have  been  designing  very  'safe'  mechanical  systems  and  have  been  using 
design  equations  in  standard  forms  to  arrive  at  such  'safe'  designs. 
Frequently  these  designs,  although  reliable,  are  costly  to  produce. 
These  higher  costs  of  'safe'  designs  are  mainly  due  to  high  material 
costs,  high  labor  costs  and  high  processing  costs.   If  suitable  opti- 
mization procedures  be  used  to  evaluate  such  designs,  it  may  result  in 
cost  savings.   Since  an  optimization  procedure  can  evaluate  many  criteria, 
the  saving  would  not  be  at  the  risk  of  the  quality. 

As  indicated  in  the  previous  chapter  the  master's  research  work 
was  started  with  a  goal  of  optimizing  some  kind  of  mechanical  design 
using  an  appropriate  optimization  technique.   The  literature  survey 
included  a  large  number  of  mechanical  designs  such  as  cold  rolling 
process,  gear  trains,  hydrodynamic  journal  bearings,  other  varieties 
of  bearings,  etc. 

The  design  of  a  hydrodynamic  journal  bearing  is  of  key  importance 
since  it  is  a  basic  part  in  rotating  machinery.   In  the  last  decade 
many  researchers  have  tried  to  optimize  different  journal  bearings. 
Each  design  is  different  because  it  depends  on  many  factors  such  as 
operating  conditions  and  behavior  of  the  parent  system  of  which  it  is 
a  part,  etc. 

Mechanical  design  is  a  multi-phase  process  requiring  constant 
decision  making  on  the  part  of  the  designer.   As  engineering  design 


has  matured  so  have  the  guidelines  and  methods  that  the  designer  has 
at  his  disposal  to  help  him  in  his  choice.   Drawing  from  his  experience 
the  engineer  is  able  to  define  variables,  a  design  objective,  and  a 
set  of  constraints  that  must  be  met  in  order  that  the  design  be  a 
workable  solution.   Thus  by  developing  corresponding  equations  a  design 
problem  can  be  stated  in  a  suitable  form  of  mathematical  programming. 

The  basis  of  hydrodynamic  journal  bearing  design  is  the  solution 
of  Reynold's  equation.   Raimondi  and  Boyd  (25)  of  the  Westinghouse 
Research  Laboratories  gave  the  solution  in  terms  of  performance  char- 
acteristic curves.  The  series  of  three  papers  gave  performance  curves 
for  different  assumptions  made  in  each  of  them.   All  of  them  discuss 
the  journal  bearing  with  the  load  at  the  center.   The  first  paper 
discusses  the  journal  bearing  with  length-to-diameter  ratio  of  one, 
constant  oil  viscosity  and  no  film  rupture.   The  second  paper  in  the 
series  discusses  the  centrally  loaded  bearing  with  length-to-diameter 
ratio  of  1/2  and  1/4.  A  problem  illustrating  the  L/D  ratio  on  journal 
misalignment  shows  that  the  shortest  bearing  is  not  necessarily  the 
one  capable  of  tolerating  the  maximum  misalignment.   The  third  paper 
discusses  the  bearing  which  accounts  for  the  film  rupture  and  perform- 
ance curves  for  different  L/D  ratios. 

Seireg  and  Ezzat  (26)  presented  an  automated  system  for  the  selection 
of  the  length,  clearance  and  lubricant  viscosity  which  optimize  the  per- 
formance of  hydrodynamic  journal  bearings  under  specified  values  or 
range  of  loads  and  speeds.   The  authors  derived  equations  from  the 
curves  given  by  Raimondi  and  Boyd  (25).   A  curve  fitting  technique  was 
used  to  arrive  at  the  functional  equations.   This  paper  discussed 


optimization  of  bearings  based  on  competitive  objectives  of  reducing 
the  bearing  temperature  and  at  the  same  time  reducing  the  oil  flow. 
The  optimization  technique  used  here  was  gradient  search. 

2.2  PROBLEM  FORMULATION 

The  usual  design  procedure  requires  mathematical  formulation  of 
the  problem  under  consideration  as  an  optimal  programming  problem. 
The  policy  can  be  as  follows; 

1.   Defining  the  system  variables  and  the  decision  parameters, 
ii.   Stating  equality  or  inequality  constraints  imposed  on  the 
design, 
iii.   Defining  the  expressions  relating  different  parameters 
governing  the  system  behavior, 
iv.   Developing  the  search  technique  best  suited  for  the  problem 
under  consideration. 
2.2.1  System  Parameters 

The  main  independent  parameters  for  the  problem  under  consideration 
are 

(D,  L,  C),  (y),  (W,  N).  S 


where 


D  ■  journal  diameter,  inches 

L  ■  bearing  length,  inches 

C  ■  radial  clearance,  inches 

y  -  lubricant  average  viscosity,  Reyn 

W  -  bearing  load 

N  ■  journal  rotational  speed,  rps 

S  ■  Sommerfeld  number  (defined  on  page  11) 


These  parameters,  as  grouped,  describe  the  bearing  geometry,  oil 
characteristics,  and  load  specifications,  respectively.   In  journal 
bearing  design  the  parameters,  D,  N  and  W  are  assumed  to  be  known. 
Hence,  the  design  parameters  left  are,  L/D,  C,  u.  Figure  2.1  shows 
the  relationships. 

Now  the  constraints  on  the  design  parameters  are  defined  as 
follows; 

K   -  h  * 
0    min 

t    <  t 
max    maxn 

P    -  P 
rmax   rmaxn 

>  "  Mmin 

L.   iLf  L 
min        max 

where,  h_  -  minimum  oil  film  thickness,  inch 

t    ■  maximum  oil  film  temperature,  °F 
max 

p    *  maximum  oil  film  pressure,  psi. 

These  constraints  are  dictated  by  the  quality  of  machining,  the  char- 
acteristics of  the  material-lubricant  pair,  and  available  space. 

2.2.2  Governing  Equations 

The  governing  equations  are  derived  from  the  curves  given  by 
Raimondi  and  Boyd  (25)  in  their  papers.   A  curve  fitting  technique 
is  used  to  derive  the  equations  (26).   They  are  used  for  the  calcula- 
tions of  the  temperature  rise,  maximum  oil  film  pressure,  oil  flow, 
frictional  loss,  etc.   For  example 
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Fig.  2.1     Bearing   Geometry. 
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for  S  1   0.5 

and  0.25  i   L/D  i   0.5 

we  have  n  q77 

hQ  -  1.585-  C  (L/D)0'913  (S)0.655(L/D)°-9 


Kt   .    0.5     .  W..0.695/(L/D)°-139 
Lt       o  374    ( S' 


P 

max 


0.76  (L/D)0'62  (S)0'24 


M^-  0.128  (L/D)0'048  (S)0'1  (L/D)0'47 


f(R/c)._12A^(s)0.62/(L/D)°-1035 
(L/D)0'41 

Similarly,  other  equations  for  different  ranges  of  (L/D)  and  S 
are  the  same  in  structure  but  have  different  values  of  the  constants. 
Here  S  is  known  as  Sommerfeld  number  and  is  defined  as 

S  -  (R/C)2  *j,   where  R  -  |. 

The  rest  of  the  mathematical  relationships  between  the  design  variables 
for  different  values  of  constants  are  given  in  the  Appendix  A. 

2.2.3  Design  Criterion 

The  designer  should  develop  a  design  criterion  which  accurately 
describes  the  designer's  objectives.   In  the  bearing  design  problem 
many  criteria  can  be  envisioned.  Some  of  them  can  be  stated  as  follows. 
Minimizing  the  maximum  temperature  rise  in  the  bearing,  minimizing  the 
oil  flow  required  for  accurate  lubrication,  minimizing  the  frictional 
loss,  etc.  The  objective  can  be  composed  of  the  multitude  of  the  above 
mentioned  factors. 
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2.3  COMPETING  OBJECTIVE  FUNCTIONS  IN  OPTIMIZATION 

A  competing  objective  function  is  the  one  in  which  the  components 
are  dependent  on  each  other  and  by  decreasing  one  we  have  to  increase 
the  other  and  the  overall  objective  may  be  to  optimize  the  sum  of  the 
both  at  a  time.  For  example,  in  the  bearing  design  problem  we  want 
to  decrease  the  rise  in  bearing  temperature  along  with  the  decrease  in 
oil  flow  in  the  bearing.  Now  for  minimizing  bearing  temperature  rise 
we  have  to  increase  oil  flow,  but  our  objective  also  consists  of  minimiz- 
ing the  oil  flow  in  the  bearing.   Hence  we  have  to  compromise  at  some 
stage  by  weighing  the  relative  importance.   There  are  various  methods 
to  accomplish  this. 

One  approach  to  multiple  objective  problem  does  not  include  any 
formal  optimization.  It  requires  that  the  designer  have  sound  knowledge 
about  the  problem.   In  this  the  designer  selects  a  candidate  design  and 
uses  a  computer  to  do  the  analysis  required  to  determine  the  behavior 
of  the  design.   Then  interactively  the  design  is  altered  by  the  designer 
until  a  'fair'  compromise  design  is  obtained.   This  selected  design  is 
not  necessarily  optimum,  and  the  designer  has  a  little  insight  in  the 
sacrifice  of  one  objective  for  the  improvement  of  the  other. 

Another  approach  uses  an  optimization  algorithm  to  find  the  'best' 
design  variable  changes  to  determine  a  new  estimate  for  the  optimum 
design.   This  interactive  procedure  continues  until  a  'good'  design  is 
found.   In  this  approach  the  designer  participates  directly  in  optimiza- 
tion process  and  as  a  result  he  obtains  further  understanding  of  the 
problem.   The  optimization  procedure  directs  the  designer  continually 
towards  optimum  design.   Sometimes,  the  designer's  bias  may  prevent 
considerations  of  all  possible  compromises  between  the  objectives. 
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Another  approach  is  simply  to  consider  all  but  one  multiple 
objectives  as  constraints  and  one  as  objective.  Thus  it  reduces  the 
multiple  objective  problem  to  the  single  objective  one.  This  approach 
requires  difficult  decisions  regarding  the  selection  of  constraints 
and  constraint  values. 

One  more  approach  to  multiple  objective  problem  involves  the 
selection  of  an  optimum  linear  combination  F  of  the  objectives.   The 
function  F  can  be  defined  as 


F  -  fx(u)  +  w2f2(u)  +  ...  +  wnfn(u). 

The  selection  of  the  best  weighing  factor  w,   depends  on  many  different 
factors  and  also  on  the  particular  type  of  problem.  This  particular 
type  of  approach  has  the  advantage  of  considering  only  optimum  designs 
but  it  may  be  difficult  and/or  expensive  to  determine  the  best  weighing 
factors. 


2.4   SEARCH  METHOD 

The  choice  of  search  method  adopted  for  the  automated  design  should 
be  very  careful.   It  should  suit  the  design  domain  and  the  criterion 
under  consideration.   Due  to  the  complex  structure  of  the  design  domain 
in  the  case  of  bearing  design,  Che  search  method  should  also  allow  for 
starting  points  which  may  violate  the  constraints.   Seireg  and  Ezzat  (26) 
and  Bartel  and  Marks  (2)  have  used  the  gradient  search  technique  for  the 
solution  of  the  problem.   But  Eason  and  Fenton  (10)  suggested  that 
pattern  search  and  simplex  search  are  better  than  the  gradient  tech- 
niques using  secant  derivate  approximation.   Hence,  we  chose  simplex 
pattern  search  (11,23,28)  for  the  solution  of  the  optimum  bearing  design. 
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After  repeating  the  problem  stated  before  we  wanted  to  extend  the 
static  problem  to  dynamic  one.  But  it  was  found  difficult  to  do  that. 
The  reason  was  the  unavailability  of  a  complete  analytical  solution 
to  the  Reynold's  equation.  The  analytical  form  was  needed  for  the 
extension  of  the  problem.  Even  very  early  papers  from  the  thirties 
and  forties  had  only  numerical  solutions  to  the  Reynold's  equation, 
either  in  the  form  of  tables  or  curves.   Seireg  and  Ezzat  (26)  arriv- 
ed at   algebraic  relationships  which  were  found  by  curve  fitting 
techniques',  but  they  also  have  constants  in  the  form  of  absolute  values 
and  hence  it  would  not  be  advisable  to  use  these  relationships  without 
completely  knowing  them. 
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CHAPTER  3 
STATE  INCREMENT  DYNAMIC  PROGRAMMING 

3.1  GENERAL 

One  of  the  most  important  optimization  techniques  developed  in 
last  three  decades  is  Bellman's  dynamic  programming  (3,4,5).   It  is 
capable  of  solving,  at  least  in  principle,  many  important  and  difficult 
optimization  problems  irrespective  of  their  nature,  linear  or  non- 
linear.  However,  because  of  the  extremely  large  high  speed  memory 
requirement,  only  relatively  simple  problems  have  been  solved  on 
existing  computers.   It  has  been  found  by  experience  that  the  method 
works  satisfactorily  until  the  problem  has  three  state  variables.  For 
more  than  three  state  variables  it  is  not  a  very  accurate  method  and 
sometimes  memory  overflow  occurs.   Bellman  calls  this  difficulty  'the 
curse  of  dimensionality. ' 

After  realizing  the  dimensionality  difficulty  in  dynamic  program- 
ming many  other  numerical  methods  have  been  developed.   Some  of  them 
can  be  listed  as  (21) , 

i.  polynomial  approximation 
ii.   lagrange  multiplier 

iii.   state  increment  dynamic  programming 
iv.   differential  dynamic  programming 
v.   quasilinearization  (19),  etc. 
The  first  two  of  them  trade  off  computer  time  for  the  high  speed  memory 
requirement.   Quasilinearization  linearizes  the  function  iteratively 
using  the  Newton-Raphson  method.   The  advantage  of  this  method  is  that 
one  gets  rid  of  the  control  variables;   hence  is  easy  to  solve.  Also, 
the  method  has   quadratic  convergence  whenever  the  problem  converges. 


16 


State  increment  dynamic  programming  essentially  reduces  the  high  speed 
memory  requirement  in  the  problem.   The  way  in  which  this  is  achieved 
will  be  explained  in  this  chapter.  Many  times  it  takes  more  time  than 
dynamic  programming  but  in  some  cases  reduction  in  computation  time 
has  been  also  observed.   In  a  particular  case  reduction  from  10  storage 
locations  to  100  storage  locations  has  been  obtained  (15). 

R.  E.  Larson  (15,16,17,18)  is  the  inventor  of  state  increment 
dynamic  programming.  While  working  at  Stanford  Research  Institute  in 
mid-sixties  he  developed  this  method  and  published  many  papers  and  a 
book  showing  method  and  its  applications. 

State  increment  dynamic  programming  is  based  on  the  'principle  of 
optimality'  given  by  Bellman.  As  previously  stated,  "an  optimal 
policy  has  the  property  that,  whatever  the  initial  state  and  initial 
decision  are,  the  remaining  decisions  must  constitute  an  optimal  policy 
with  respect  to  the  state  resulting  from  the  first  decision." 

The  difference  between  the  conventional  dynamic  programming  developed 
by  Bellman  (3)  and  the  state  increment  dynamic  programming  by  Larson  (15) 
is  in  the  choice  of  the  time  interval  over  which  a  given  control  is 
applied.  Conventional  dynamic  programming  uses  a  fixed  time  interval 
whereas  state  increment  dynamic  programming  determines  the  time  interval 
as  the  minimum  time  required  for  at  least  one  of  the  state  variables  to 
change  by  one  increment.  As  a  result  of  the  choice  of  time  interval,  the 
next  state  after  applying  the  control  will  lie  on  the  surface  of  an 
n-dimensional  hypercube   centered  at  the  given  point  and  with  length 
Z  (Ax.)  along  the  i   state  variable  axis.  This  property  is  used  to  reduce  the 
fast  memory  requirement.   The  entire  state- time  (X-t)  space  is  partitioned 
into  several  blocks.   Each  block  covers  some  increment  along  each  state 
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and  a  longer  interval  along  the  time  axis.   Then  the  optimal  value  of 
the  variables  is  calculated  for  one  block  at  a  time,  and  not  for  the 
entire  state  space  as  in  the  standard  dynamic  programming  algorithm. 

3.2  PROBLEM  FORMULATION 

The  optimization  problem  to  which  the  state  increment  dynamic 
programming  procedure  is  applied  is  most  conveniently  formulated  in 
the  continuous  case  over  the  interval  t.  5  t  S   tf.  Hence,  the  system 
equation  becomes  a  set  of  nonlinear  time  varying  differential  equations. 

x-f_(x,u,t)  (3.1) 

where        x_  «■  state  vector  in  n  dimensions 

u  ■  control  vector  in  q  dimensions 

£  -  stage  variable,  usually  time 

f_  ■  n  dimensional  vector  functional. 

The  performance  criterion  to  be  minimized  is  a  cost  function  denoted  by 

J.   It  consists  of  the  sum  of  an  integral  with  respect  to  a  scalar 

function  of  state  variables,  control  variables,  and  stage  variables 

and  a  scalar  function  depending  on  the  final  state  and  the  final  stage. 

Thus  tf 

J  -  l      *[x(o),  u(o),o)]do  +  *[x(t  ),  tf]  (3.2) 

i 

where       t.  -  initial  stage  (time) 

tf  ■  final  stage  (time) 

a  -  dummy  variable  for  stage 

J  ■  cost  function 

I   ■  loss  function;  cost  function  per  unit  time 

<J/  ■  final  value  term  in  cost  function. 
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Constraints  are  on  both  state  and  control  variables  of  the  form 

x  e  X(t) 

u  £  U(t)  (3.3) 

where 

X(t)  ■  set  of  allowable  states  of  the  time  t 

U(t)  ■  set  of  the  admissible  controls  at  state  x 
and  time  t. 
Assumption;  it  is  assumed  that  u(t),  t .  1  t  *  t,,  is  piecewise  constant 
over  intervals  of  length  6t.  In  order  to  implement  the  procedure  on 
a  digital  computer  the  set  of  differential  equations  is  approximated 
by  a  set  of  difference  equations. 

Thus,  x(t+At)  -  x(t)  +  fjx(t),  u(t),  t]6t  (3.4) 

and  the  change  in  performance  criterion  over  the  interval  from  t  to 
(t  +  6t)  is  approximated  by 

ft  *[x(o),  u(a),o]  da 

-  4[x(t),  u(t),  t]«t  (3.5) 

Again,  if  fit  were  fixed  to  a  value  At  then  the  computation  could 
be  done  by  the  conventional  dynamic  programming  method,  but  it  is  not  the 
case  here.   6t  is  determined  by  computations  and  is  the  basic  element 
of  state  increment  dynamic  programming. 

The  functional  equation  can  be  derived  from  the  cost  function 
based  on  Bellman's  principle  of  optimality  and  can  be  stated  as 


I(x,t)  -  Min  {£[x,u,t]«t  +  I[x  +  f (x.u.t)  t,6t  +6t]}   (3.6) 
ueU  ~ 
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3.3  CONSTRAINTS  AND  QUANTIZATION 

The  constraints  are  restricted  to  a  set  of  admissible  states  X 
and  a  set  of  admissible  controls  U.   For  example,  inequality  constraints 
of  the  form  $(x, t)  -   0  can  be  used  to  bound  the  state  variables 

8~  5  xt  i   8*  ,     for  i  -  1,2,  ...,  n.  (3.7) 

Inequality  constraints  of  the  type,  $(x,  u,  t)  -  0,  can  be  used  to 
restrict  the  control  variables 


a"  i  u,  1  a .  ,     for  j  -  1,  2,  ....  q.  (3.8) 

The  quantities  8.  and  8.  can  vary  with  t,  while  the  quantities  a.  and 

+ 
a 


.  can  vary  with  x  and  t. 


Within  the  allowable  range,  each  state  variable  x.  is  quantized 
into  a  finite  number  of  values,  N . .   It  is  convenient  to  assume  the 
quantization  in  constant  increments,  Ax..   The  result  is 


where 
and 


Xi  "  8i  +  ji  Axi  (3'9) 


j  .  ■  1,  2,  ...,  N . 

Ni  &x±   -  8  -  8~       for  i  -  1,2,  ....  n. 


The  set  of  state  vectors  for  which  each  component  has  the  form  of 
Eq.  3.9  is  called  the  set  of  quantized  admissible  states,  X. 

Although  the  control  variables  can  be  quantized  in  a  similar 
manner,  it  is  necessary  only  that  there  be  a  finite  number  of  admiss- 
ible controls.   The  set  of  admissible  controls,  U,  can  be  given  as 
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0-  {u(1),  u(2) u(o)}  (3.10) 

The  choice  of  u    e  U  depends  on  the  problem  under  consideration. 


3.4  DETERMINATION  OF  THE  TIME  INTERVAL  St 

In  state  increment  dynamic  programming  the  time  control  for  com- 
putation of  the  optimal  control  (At)  may  or  may  not  be  fixed  depending 
on  the  nature  of  the  problem.  But  the  interval  <5t  varies  with  the 
control  applied.   The  interval  5t  is  determined  as  the  minimum  time 
interval  required  for  any  one  of  the  n  state  variables  to  change  by  one 
increment.  For  example,  if  Ax.  is  the  increment  in  the  i   state 
variable  and  if  control  _u  is  applied,  then 

5t  -  Mi?  0       {■,  .     i  -rf)  (3.11) 

i«l,2,...,n   |fi(2E»u.»t)| 

where  f.(x,u_, t)  is  the  i   component  of  f_(x,u_,t),  the  system  differ- 
ential equation  vector. 

Expression  of  6t  as  in  Equation  (3.11)  is  the  basic  equation  in 
state  increment  dynamic  programming.  This  also  shows  that  the  next 
state  lies  only  Ax.  from  the  original  one.   Also,  for  the  iteration  of  the 
minimum  cost  function,  only  the  value  at  these  quantized  states  at  Ax. 
from  original  state  need    be    stored.   And  this  is  the  property 
which  reduces  the  fast  memory  requirement  by  a  considerable  amount. 

3.5  BLOCK  CONCEPT 

The  significant  amount  of  reduction  in  high  speed  memory  require- 
ment can  be  achieved  by  processing  the  data  so  as  to  obtain  the  maximum 
utilization  of  the  reduction  for  a  single  calculation.   This  is  done 
by  computations  in  units  called  blocks. 
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Blocks  are  defined  by  partitioning  the  n+1  dimensional  space  Into 
rectangular  subunlts.  Each  block  covers  an  increment  w  along  the 
x  -  axis  and  AT  along  the  time-axis.   A  particular  block  can  be  denoted 
by  the  largest  value  of  the  coordinates  that  are  contained  within  the 
block.  Figure  3.1  shows  the  block  for  a  problem  with  two  state  vari- 
ables. For  an  n  state  variable  problem  the  block  can  be  denoted  as 

This  block  contains  the  values  of  t  and  x  such  that 


(j0-l)  AT  <  t  -  ti  <  jQ  AT 


and 


where 


(j^l)  wt  Ax±  1  x±  -   &"  5  j2  w±   Ax±  (3.12) 


Jq  ™  1»   2 J_ 

J0  "  AT  *  e£  "  Ci 
j.  ■  1,  Z,  • • • »  Jj 

J±  v^   Axi  -  &±  -  &± 

1  ■  1,  2,  .. . ,  n. 

For  a  two  dimensional  (n«2)  problem  each  block  is  a  three  dimensional 
rectangular  solid,  as  shown  in  Figure  3.1,  which  has  two  axes  for  the 
stage  variable. 

3.6  COMPUTATIONS  WITHIN  THE  BLOCK 

The   computational  procedure  assumes  that  the  next  state  is  within 
the  block  under  process.   In  the  general  case  the  time  interval  is 
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Fig.    3.1  Block  Diagrao  For  Two   Stace  Variables  Cas< 
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divided  into  smaller  increments  At.   The  set  of  quantized  times  are 
determined  as 

t  -  tQ  +  (j0-l)  At  +  SAt  (3.13) 

where 

s  »  0,  1,  2,  ...»  S 

and 

S  At  -  AT 
Typical  values  of  S  range  between  5  and  15.   Then  the  optimal  control 

is  computed  at  each  quantized  state  xeX  for  each  quantized  stage  t 

(k) 
as  given  in  Equation  (3.13).  Then  each  admissible  control  II    £  U  is 

applied.  Over  each  control  the  time  over  which  it  is  applied  is  deter- 
mined as  in  Eq.  (3.11)  as 

««w-£.a -<l      V  !>•  «•"> 

f1(x,uv   ,t) 

With  this  state  and  time  the  optimal  cost  function 

I(x(k>,  t+6t(k)) 

is  computed  by  interpolation  in  (n-1)  state  variables  and  time  using 

previously  calculated  values  at  quantized  state  and  times  t+At, 

t+2At,  ....   If  the  control  is  such  that  none  of  the  state  variables 

change,  i.e.,  if  f_(x,  u(k) ,  t)  -  0,  then  x(k)  -  x  and  5t(k)  is  set 

equal  to  At.  The  resulting  next  states  for  a  one-dimensional  example 

are  shown  in  Figure  3.2,  where 

„   ,  (1)    (2)    (3)    (4)    (5), 
U  -  {u   ,  u   ,  u   ,  ux   ,  u   } 

values  of  optimal  cost  function  are  known  at  the  points  indicated  by 

small  circles. 
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Once  the  optimal  cost  is  found  out  for  each  of  the  next  states, 

(k) 
the  cost  of  each  control  over  time  6t    is  found  by  evaluating  the 

cost  function  per  unit  time 
t(x,  u°°,  t). 


The  optimal  cost  and  optimal  control  at  x,t  are  then  calculated  by 
using  the  principle  of  optimality.  The  result  is 


Kx.t)  -  £  2k  U(x,u(k),t)6t(k)  +  T(x(k),  t  +  6t(k)},   (3.16) 


k  -  arg  £t2t##.ik.  U(x,u(k\t)6t(k)  +  T(x(k),t  *  6t(k)>,    (3.17) 
G(x,t)  -  u(k). 


3.7   INTERBLOCK  TRANSITION 

This  can  be  divided  into  two  parts.   One  is  the  transition  into 
previously  processed  blocks,  and  the  other  transition  is  into  previously 
not  processed  blocks. 

The  simplest  case  of  the  previously  computed  block  transition  is 
the  one  in  which  the  boundary  is  common  between  the  two  blocks.   In 
this  case  there  is  no  need  to  find  new  optimal  points  on  the  boundary. 
However  minimum  costs  at  these  states  are  stored  in  the  high  speed 
memory  and  then  they  are  used  in  interpolation  formulae  for  minimum 
costs  at  next  state.   The  storage  of  these  values  allows  transition 
from  the  block  currently  being  computed  to  the  previously  computed 
block.   As  long  as  the  values  of  minimum  costs  on  such  a  boundary  are 
available,  transitions  of  this  type  to  previously  computed  blocks  can 
be  made  without  constraint. 
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In  a  system  to  which  state  increment  dynamic  programming  is  applied 
there  is  generally  some  prior  knowledge  about  the  behavior  of  optimal 
trajectories.  The  knowledge  of  direction  of  the  optimal  trejectory  is 
called  the  perferred  direction  of  motion.   Then  according  to  the  pre- 
ferred direction  the  order  of  processing  of  the  blocks  is  determined. 
The  processing  order  is  inverse  of  the  preferred  direction  to  achieve 
the  transition  to  previously  computed  block.  This  helps  in  saving  a 
significant  amount  of  computational  time. 

The  second  of  the  interblock  transitions  is  the  transition  to 
blocks  not  previously  computed.   The  simplest  technique  is  to  exclude 
all  controls  which  result  in  such  a  transition  during  the  computation 
of  a  block,  but  to  consider  such  a  transition  after  both  the  blocks 
have  been  computed.   Therefore  it  will  be  clear  that  in  the  case  of 
computation  of  boundary,  the  controls  which  take  the  next  state  in  the 
block  that  is  not  yet  computed  are  not  allowed.   However,  when  the 
later  block  has  been  computed  such  controls  are  applied  at  the  boundary 
at  the  least  value  of  t  within  the  block.   Then  the  minimum  costs  are 
found  for  all  points,  the  points  on  the  boundary  as  well  as  the  points 
within  the  later  block.   If  one  of  these  costs  is  less  than  the  existing 
cost  then  it  replaces  that  cost  and  becomes  the  new  minimum  cost,  and 
the  corresponding  control  becomes  optimal  control. 

A  more  accurate  procedure  for  allowing  these  transitions  is  to 
extrapolate  the  minimum  cost  function  into  the  not  yet  computed  block. 
In  general,  extrapolation  procedures  are  less  accurate  than  interpol- 
ation procedures.   However,  the  extension  of  state  by  extrapolation  is  at 
most   Ax.  in  the  x.  direction  and  hence  the  error  is  strictly  bounded. 
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Fig.  3.3      Linear  Interpolation  In  One  Dimensional  Case. 
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Moreover  at  every  At  interval  along  t,  the  minimum  costs  are  recomputed 
along  the  boundary  and  that  does  not  allow  the  error  to  accumulate  due 
to  extrapolation. 

If  extrapolation  does  not  provide  adequate  results,  then  the 
recomputation  of  the  results  along  the  boundary  can  be  done.   In  this 
case  the  optimal  control  and  the  minimum  cost  along  the  boundary  are 
recomputed  when  the  block  in  which  the  extension  was  made  is  processed. 
In  general,  the  increase  in  computing  time  and  high  speed  memory  re- 
quirement are  not  worth  the  slight  increase  in  accuracy. 

A  better  alternative  is  to  precompute  some  results  in  order  to  know 
the  preferred  direction  of  motion.   If  this  is  done  then  fewer  of  the 
optimal  trejectories  will  go  in  the  non-preferred  directions  and  hence 
the  results  of  extrapolation  procedure  will  be  used  less  often  in 
computing  minimum  cost. 
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CHAPTER  4 
AN  ADVERTISEMENT  PROBLEM 

The  computational  aspects  of  the  state  increment  dynamic  program- 
ming will  be  discussed  in  this  chapter.  The  problem  used  to  illustrate 
the  method  is  the  one  of  inventory  and  advertisement  system  with  two 
state  variables  and  one  control  variable. 

A.l  DEVELOPMENT  OF  THE  MODEL 

A  diffusion  model  for  advertisement  was  originally  developed  by 
Teichroew  (29).  The  model  discussed  here  is  an  extension  of  his  model. 
Consider  a  particular  sales  system  where  in  a  group  of  people,  only 
certain  members  possess  a  particular  piece  of  information.  The  group 
size,  i.e.  the  number  of  people  in  the  group,  is  assumed  to  be  constant. 
Also,  the  diffusion  of  information  occurs  only  through  personal  contact. 
The  number  of  'contacts'  made  by  an  'average'  person  in  an  arbitrary 
unit  of  time  and  is  given  as  a  contact  coefficient.   It  is  assumed  the  same 
for  all  the  people  in  the  group.   A  contactee  receives  the  information 
only  if  he/she  does  not  already  have  it;  otherwise  the  contact  is  wasted 
as  far  as  increasing  the  number  of  informed  persons  is  concerned. 

Let      Q(0)  •  Q~  ■  number  of  informed  persons  at  time  t . . 

N  ■  total  number  of  people  in  group 
C  ■  contact  coefficient,  the  number  of  contacts 
made  by  one  informed  person  per  unit  of  time 
Q(t)  ■  number  of  informed  persons  at  time  t. 
Therefore  ^*jj —  -  the  fraction  of  informed  persons  at  time  t,  and 

1  -  jj —  "  the  fraction  of  uninformed  persons  at  time  t. 
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The  contacts  made  during  a  time  interval  dt  can  be  given  by 

C  Q(t)  dt. 
c  ^ 

The  increase  in  the  total  number  of  informed  people  during  short  time 
interval  At  is  found  by  multiplying  the  number  of  contacts  by  the 
proportion  of  the  uninformed  persons  because  an  increase  in  the  in- 
formed persons  can  be  caused  only  by  the  proportion  of  the  uninformed 
people  of  the  group.  Therefore 


dQ(t)  -  Cc  Q(t)  dt  (1  -  SM) 
and 

mi .  Cc  Q(t)  (1 L  M£i)  \  (4.i) 

Suppose  the  company  thinks  that  it  can  influence  the  number  of 
contacts  by  spending  money  on  advertising  and  that    the  rate  of 
contacts  by  each  informed  person  can  be  increased  by  an  amount  A  per 
unit  time,  then 

^M  .  [cc  +  A(t)]  Q(t)(l  -  ^-)  .  (4.2) 

If  each  informed  person  purchases  C  units  of  company's  product  and 
the  sales  at  the  time  t  can  be  denoted  by  S(t),  then 

S(t)  -  Cq  Q(t)  (4.3) 

For  simplifying,  if  C  can  be  taken  to  be  unity,  then 

S(t)  -  Q(t)  (4.4) 

Substituting  S(t)  for  Q(t)  in  Equation  4.2,  we  have 

4|M  -  [Cc  +  A(t)]  S(t)(l  -  -^O  .  (4.5) 
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The  rate  of  change  of  the  company's  inventory  is  given  by 

*$*■'   P(t)  -  S(t)  (4.6) 

where  P(t)  ■  production    by   time  t.  The  production  rate  is  assumed 
to  be  a  linear  function  of  time  and  can  be  given  by 

P(t)  -  a  +  b  t  (4.7) 

where  a  and  b  are  constants. 

The  objective  can  be  defined  at  this  stage.   The  management  desires 
to  maximize  profit.   In  this  problem  the  profit  is 

Profit  ■  Sales  revenue  -  Inventory  carrying  cost  -  Advertisement  cost. 

In  mathematical  form  it  can  be  written  as 

J  -  /  f[C  S(t)  -  CT(I  -  I(t))2  -  C.S(t)  A2(t)]dt        (4.8) 
u  l  m  A 

where     J  -  net  total  profit 
C  ■  sales  revenue 
I  ■  capacity  of  storage  of  inventory 

CT  ■  inventory  carrying  cost 

C.  -  advertisement  cost 
A 

We  have  explained  and  derived  the  mathematical  model  in  differ- 
ential form.   But  here  we  are  solving  the  problem  by  using  state  Increment 
dynamic  programming,  which  requires  discrete  form  of  the  problem.   There- 
fore we  shall  transform  the  above  differential  equations  into  difference 
equations. 
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The  system  variables  are  inventory  at  time  t  and  sales  by  time  t. 
In  difference  equations  they  can  be  given  as 

I(t  +  At)  -  I(t)  +  [P(t)  -  S(t)]  At  (4.9) 

and 

S(t  +  At)  -  S(t)  +  S(t)[Cc  +  A(t)][l  -  SJ&-]  (4.10) 

By  inspection  of  the  equation  4.10  one  can  see  that  if  it  be 
kept  in  the  same  form  then  S(t  +  At)  will  be  greater  than  N  at  one 

stage.  This  cannot  be  allowed.  Therefore  a  little  modification  is 

S(t) 
done  in  it.  The  term  (1 —-1-)   has  been  replaced  by  the  term 


(1 ■ — - -)  .  The  new  equation  is 


N 

S(t  +  At)    -  S(t)   +  S(t)    [Cc  +  A(t)][l  -  S(t  +  At>] 

-  S(t)   +   [C     +  A(t)]    S(t) 

C 

-  [Cc  +  A(t)]    S(t)    •    S(t  +  At) 

C     +  A(t) 
S(t  +  At)    [1  +  {-2— - }    •    S(t)    •   At] 

-  S(t)    [1  +  {Cc  +  A(t)}] 
S(t)    [1  +  {C     +  A(t)}   At] 

s«  (t  +  At) chrm —  <*•"> 

1  +  S(t){-£-^ }    •    At 

Now  we  have  to  change  the  profit  function  into  difference  equation 

form.   The  result  is 

(n+D  At 

/     [c  S(t)  -  Cjtt^  -  I(t)T  -  CA  S(t)  A*(t)]  •  dt 
nAt 

-  [c  S(t)  -  Cx(Ia  -  I(t))2  -  CA  S(t)  A2(t)]At 

-  Ux,  u(t),  t)  dt  (4.12) 
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which  is  a  current  stage  profit  equation  in  terms  of  state  and  control 
variables.  The  complete  functional  equation  for  state  increment  dynamic 
programming  can  be  written  as 


I(x,  t)  -  Min  U(x,  u(t),  t)  5t 
ueU 

+  I[x  +  f  (x,  u(t),  t)  fit,   t  +  6t]}.  (4.13) 


4.2  DEFINITION  OF  PROBLEM 
The  goal  is  to  maximize 

Kx,  t)  -  *JU  U(x,  u(t),  t)  •  6t  +  I[x  +  f(x,  u(t),  t) 

+  I[x  +  f  (x,  u(t),  t)  6t,  t  +  6t]} 

and 

H(x,  u(t),  t)  •  5t 

-  [c  •  S(t)  +  C_  (I  -  I(t))2  -  C.  S(t)  A2(t)]  «t        (4.14) 
l   m  A 

subject  to 

P(t)  -  a  +  bt  (4.15) 

I(t  +  At)  -  I(t)  +  [P(t)  -  S(t)]  5t  (4.16) 

and 

S(t)  [1  +  {C  +  A(t)}  At] 

s-  (t  +  At) c-TTiT) '  (4'17) 

1  +  S(t)  (-^-^ }  •  At 

4.2.1  Numerical  Aspects 

In  order  to  solve  this  problem  the  constants  were  assumed  to  have 
following  values 
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a  ■ 

70 

N  - 

150 

b  - 

100 

1     - 

■ 

50 

Ci" 

2 

6i" 

0 

C  - 
Ca" 

20 

0.5 

Cf  - 

1 

C     - 
c 

0.15 

Initial  conditions  are: 

1(0)  -  20 

S(0)  -  20 
The  maximum  amount  of  advertising  at  any  time  has  been  restricted  to  a 
value  of  6.   This  means  that  A(t)  1  6.   This  is  the  constraint  on  the 
control  variable. 

4.3   SOLUTION  BY  STATE  INCREMENT  DYMAMIC  PROGRAMMING 

From  the  problem  solution  by  quasilinearization  (27),  it  is  known 
that  the  value  of  both  the  state  variables,  x(t)  and  S(t)  respectively, 
increase  with  the  time.   Hence  this  will  be  the  preferred  direction  of 
motion  for  them. 

Using  the  block  concept,  we  have  three  dimensional  block  of 
Ax  X  S  X  AT.   The  time  interval 
AT  -  tf  -  tQ  -  1, 

is  divided  into  ten  equal  parts  giving 
At  -  0.1. 
Since  the  preferred  direction  of  motion  of  x(t)  and  s(t)  is  such 
as  to  increase  with  time,  we  shall  first  process  the  blocks  with  the 
largest  values  of  x,  s  and  t.   First  of  all,  in  this  block,  x  and  s 
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are  kept  constant,  and  optimal  function  values  for  all  ten  time  incre- 
ments are  found  out.   Then  x  is  kept  constant  and  s  is  lowered  by  one 
increment  and  then  optimal  function  values  are  found  for  each  stage 
(time  interval) .  Then  x  is  lowered  by  one  increment  and  above  procedure 
is  repeated.   This  goes  on  till  the  optimal  function  is  evaluated  for 
all  values  of  x,  s,  and  t. 

As  shown  in  Figure  4.1,  the  adjacent  block  will  have  a  common 
boundary  with  the  original  one.   The  just-counted  optimum  values  at 
the  original  block  boundary  will  become  the  initial  values  for  the  new 
block.   These  values  will  be  used  to  find  the  optimum  values  of  the 
next  block. 

4.4  PROCEDURE  FOR  OBTAINING  STARTING  VALUES  FOR  THE  FIRST  BLOCK 

To  get  good  accuracy  by  state  increment  dynamic  programming,  the 
average  value  of  the  time  interval  6t  required  to  change  any  of  the 
states  by  one  increment  should  be  close  to  At.   If  it  is  large  or  small 
compared  to  At  the  accuracy  is  decreased  because  of  inaccurate  approx- 
imations due  to  the  interpolations  or  extrapolations  performed.   In 

Ax   Ax 
other  words  —  -  -r-  -  1.   Such  a  condition  when  Ax  is  small  is  illus- 
trated in  Figure  4.2. 

In  this  figure  the  optimum  values  at  points  p.  and  p.  are  known. 
The  optimum  value  at  point  pQ  is  to  be  found  from  these  two  values. 
Since  Ax  is  very  small,  nAx  will  be  increased  by  one  increment  to 
(n+l)Ax  on  application  of  control  z  for  a  very  short  time  interval  St. 
As  a  result  the  point  p.  is  at  a  far  distance  from  points  p1  and  p„. 
This  will  result  in  an  inaccurate  optimal  value  of  p,  based  on  extra- 
polation. 


36 


4  a* 


3£^ 


3A- 


2.A-X 


2A* 


A-* 


T 


Fig.  4.1      Processing  With  Too  Snail  uX. 
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Cn*-0  a,x. 


TliX    P 


*»-  *r 


Fig.  4.2      Order  Of  Processing  The  Blocks. 
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Since  the  state  variable  increases  with  time  the  width  of  time 
block  wAx  is  taken  equal  to  Ax  and  hence  w  ■  1. 

Figure  4.1  shows  in  simplified  manner  the  order  of  processing 
the  blocks. 

The  computer  program  in  FORTRAN  language  with  a  sample  printout 
is  shown  in  Appendix  A.   It  was  solved  on  the  ITEL  AS/5  computer.  The 
flow  chart  for  the  state  increment  dynamic  programming  is  also  shown 
in  Appendix  B. 

The  results  for  the  initial  condition  were  interpolated  manually 
and  are  given  in  Table  1.  The  plots  of  the  results  for  each  visuali- 
zation are  also  given  in  Figures  4. 3-4. 7. 

4.5  RESULTS 

The  optimal  profit  in  this  problem  was  J  ■  891.00  and  the  optimal 
initial  and  final  values  are 

1(0)  -  20 

S(0)  -  20 

A(0)  -  6.0 

1(1)  -  50.0 

S(l)  -  123.22 

A(l)  -  0.0 
From  Table  1  we  can  see  the  advantage  of  using  state  increment 
dynamic  programming.   Since  we  are  using  the  principle  of  optimality 
we  get  the  optimal  values  of  each  stage  for  all  parameters.   Thia  is 
done  by  dividing  the  solution  space  into  a  number  of  grid  points  and 
evaluating  optimal  values  at  each  grid  point  where  at  least  one  of  the 
state  variables  changes  its  state. 
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Fig.    4.3  Inventory  Level  For  A  Ten   Stage  Process. 
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Fig.    4.6  Advertisement   For  A  Ten  Stage   Process. 
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The  problem  was  solved  on  ITEL  AS/5  computing  facility  at  Kansas 
State  University  on  WATFIV  compiler.   It  took  about  3.79  min.  to  get 
the  overall  results. 

4.6  DISCUSSION 

From  Table  1  and  Figures  4.3  to  4.7  we  can  see  that  the  different 
parameters  increase  with  the  time  and  so  does  the  profit. 

The  production  rate,  being  a  linear  function  of  time,  increases 
with  the  time,  while  other  parameters  like  inventory  and  sales  also 
increase  with  the  time.   The  amount  of  advertisement  decreases  with 
time  and  eventually  becomes  zero.   Since  the  sales  increase  with  time 
and  advertisement  decrease  with  time,  the  profit,  as  would  be  expected, 
should  increase  with  the  time.   This  easily  can  be  visualized  from  the 
plots  in  above  mentioned  figures.   Also  the  production  is  a  linear  func- 
tion of  time  and  hence  should  be  maximum  at  the  final  stage  which  would 
give  the  maximum  profit. 

As  has  been  mentioned  before  the  problem  with  the  same  model  has 
been  solved  by  P.  Shah  (26).   The  optimization  technique  he  used  was 
quasilinearization.   Therefore  the  best  way  to  evaluate  the  state 
increment  dynamic  programming  would  be  to  compare  our  results  with  his. 

Since  the  constants  used  here  are  different,  we  can  not  expect  the 
same  results  as  his.   But  the  trend  of  the  optimal  values  for  different 
stages  can  be  compared.   It  is  found  that  his  results  and  ours  match 
very  well  and  follow  the  same  pattern.   The  quasilinearization  took 
about  3.72  minutes  to  get  the  completely  converged  solution.   State 
increment  dynamic  programming  took  3.79  minutes  to  get  the  complete 
solution. 
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The  great  advantage  with  SIDP  is  that  we  got  optimal  values  of 
objective  function  at  all  grid  points  in  the  solution  space.  Thus  if 
one  is  interested  in  seeing  the  optimal  values  at  different  stages  or 
states,  can  get  directly  from  tables.  This  is  helpful  in  case  of 
constraint  changes  on  stage  and  state  variables.  One  does  not  need 
to  go  through  the  whole  tedious  procedure  again  and  again.  Thus  little 
increase  in  computing  cost  can  be  justified. 

The  same  kind  of  results  with  probably  a  little  more  accuracy 
could  be  obtained  by  the  standard  dynamic  programming  algorithm.   But 
the  'dimensionality'  difficulty  restricts  the  application.  For  example 
66  fast  memories  are  required  for  this  problem  when  solved  by  SIDP  while 
1200  fast  memories  would  be  required  for  solution  by  standard  algorithm. 

As  for  accuracy,  we  do  not  claim  that  the  results  are  the  most 
accurate  ones  by  this  method.   The  reason  is  the  average  time  interval 
over  which  control  is  applied  should  be  about  the  same  order  of  magnitude 
as  the  fixed  time  interval  to  get  very  accurate  results.   But  the  time 
interval  over  which  the  control  is  applied  is  divided  by  the  state  vari- 
ables and  the  control  variables.   If  one  of  the  state  variable  changes 
much  more  rapidly  than  the  other,  then  the  time  interval  for  the  appli- 
cation of  the  control  for  that  particular  variable  will  be  the  least 
and  hence  it  will  go  on  changing  all  the  time.   This  keeps  the  other 
state  variables  unchanged,  and  the  problem  moves  in  a  plane  rather  than 
space.   This  brings  in  accuracy  in  the  results  due  to  interpolations. 

Moreover  SIDP  is  a  fairly  new  method.   There  have  not  been  very 
much  researches  on  it  nor  computational  experiences.   Some  of  the 
assumptions  make  its  use  restrictive.   One  of  them  is  mentioned  above 
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while  the  other  one  is  movement  of  the  objective  in  the  preferred 
direction.  We  do  not  have  any  answers  for  the  consequences  of  moving 
in  non-preferred  direction. 

In  spite  of  these  restrictions,  SIDP  still  is  a  versatile  method 
and  can  be  used  very  successfully  where  the  standard  algorithm  does 
not  work  very  well.   It  has  extremely  good  potential  and  more  develop- 
ment can  make  it  very  powerful  technique. 
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APPENDIX  A 
Mathematical  expressions  for  journal  bearing  operative  characteristics: 

1.  S  <  0.15 

i)  0.25  <  L/D  <  0.5 

hQ  -  1.585  C(L/D)°-913  (S)0.655(L/D)0-0922 


At  -     V,-4  P(S)0'^/(L/D)0-139 
(L/D)U*J/* 


2-  -0.76  (L/D)0'62  (S)0'24 


P 

max 


^  -  0.128(L/D)°'048  (S)°-1(L/D>0,47 


+  (R/C)-_12S6   (S)0.62/(L/D)°-1035 
(L/D)0-*1 


ii)  0.5  £  L/D  i  1 

hQ  -  1.84C  (L/D)1*13  (S)0.731(L/D)°-252 


At--0^3rT,P(S)0-56/(L/D)0*302 
(L/D)0'62 


f--0.76  (L/D)0'62  (S)0.294(L/D)0-292 
max 


SgSl-  0.128  (L/D)0'048  (S)0.06/(L/D)0-212 


f(R/c)-_11^8    (S)0.62/(L/D)°-1035 
(L/D)0,503 
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2.     Si  0.15 


i)     0.25  5  £  <  0.5 


hQ  -  1.035  C    (L/D)°-673    (S)0.33/(L/D)°-2 

a.  0.695        ,_,0.875(L/D)°-042 

At  "  0~214    (S) 

(L/D)U,Zi4 


P      -0.76   (L/D)0'62  (S)0'24 


P 
max 


^-0.128    (L/D)°-048    (S)0-1(L/D>°*47 


f(R/L)   -       16'85318    (S)0»^^)0t087 
(L/D)0*318 


ii)     0.5  5  L/D  <  1 


hQ  -  0.95  C    (L/D)0'556    (S)°-375 

A.  0.695        ,_,0.875(L/D)0-042 

At  "  (T21A    (S) 

(L/D)U*^ 


1    S^S 
/--0.55    (L/D)0'1535    (s)0'083'^ 

max 


0    212 
^k-  0.128    (L/D)0-048    (S)°-06/  "■"» 


f(Wc>  — ^jj-  (S)0-922  'L/D'°-°87 

(L/D)0,1^7 
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APPENDIX  B 

//    EXEC    WATrlV 

//SYSIN  0D    * 

SJ03  ,TIM5*(5,  ),PAGES=250 

DIMENSION    S8(5l,5L  )»SP(51,5l) 

0X  =  5 

0R  =  5 

0A=,2 

OT=.l 

NT=10 

NX  =  2L 

NA  =  3l 

NR  =  30 

PI=50 

AN=150.2 

48*70 

3  =  100 

C=2 

•^20 

CI*.  15 

CA=0.5 

XMAX=100 

PR5S=-933S88 

•\JRR=NR  +  1 

NXX=NX+l 

UM<=WJ*0      • 

N8:.0C'<=0 

MS=NT 

100  FORMAT  (IHl, 'STAGE  NO       TIME      X(N-l)      RCN-1)      X(N 
I    P(N)        P(N)  A(N)       PROFIT1) 

101  F0P..M4T  (IH  ,5X»I2t8(4X»F7.2)  J 

102  -ORM^r  ( IH  , 'BLOCK  NO  =  ',13) 
PRINT  100 

00    7  1    K=i,51 
00    7L    L=l,5i 
SBK,L)=0 
71    SP(K,L>=0 

00    10    <X=L,NX 
X^,NX-<X 

x=x*ox 

00     U    <R=l»NR 

}=NRR-KR 

IR  =  R 

R=R*DR 

r=i 

^AB+B^M  r-OT  ) 
^Bl  DC.<=NBI  CCK  +  L 
OR  INT    102»NBI.0C< 
00    2  5    <A=1,NA 

a=<a-i 

A=A*OA 

<n  =  X  +  {  P-*  M!OT 

ROa(R*{ l.  +  (C  +  A )*0T )  )/( l.+R*<C  +  A)*DT/AN) 
Ii=  (X0.f.r..00l  )G0  TO  25 
SMI=W»(R*r-(  PI-X  )=»*?*(-.  I -C  A* A** 2 *R  )*0T 
li=  (  S'iEW.i.r.PRES  )  GO  TO  25 
PR5S=SN£W 
A  4  =  4 
XX  =  XO 
RR  =  RO 
-  25  CONTINUE 


51 


80 


42 


90 
91 


51 


22 


81 


43 


11 


31 


IF  (LI 

LIN6N0 

PRINT 

PRINT 

LIN5NC1 

SPfMSt 

PR6S=- 

NS=NS- 

r*NS*o 

P=AR+B 
00  22 
A=KA-l 
4= 4*0 A 
U  =  P-R 

lc  (U. 

06i.  rx  = 
oo  ro 

OELTXs 
OELTRa 
f?  (06 

06LT=0 
06LR=0 
XO  =  X-M 
RO=(R* 
Sl*(R* 
Ri  =  SB( 
R2=SR( 
S2=R1- 
SN6W=S 
GO  TO 

06)  r=o 

06L*=0 

xn=x-i-( 

ROa{R* 

Sl  =  { ** 
Xl*SP( 
X2  =  S=>( 
S?sXi- 

SNEWsS 
IFiSNc 

^R6S=S 
AA  =  A 
XX  =  XO 

RR  =  RO 

COM  TIN 

IF  (LI 

LINENG 

PRINT 

PRINT 

LIN6N0 

SPINS, 

fF(NS. 

oo  ro 

r=i 

NS=NT 

p«es«- 

CONTIN 
00  31 
00  31 
SRILNS 


N6N0.LT. 45)  GO  TO  80 

=  0 

100 

10i,NS. TtX,R,XX,RR,P,AA,PRES 

=LIN6N0+l 

[R  )=PR6S 
988888 

I 

T  

*(T-DT) 
<A=l,NA 


60 
AR 
91 

99 
AR 
IT 
EL 

6! 

(l 

NS 
NS 

(R 

l  + 

52 

6! 
PI 

(1 


..0  )  GO  TO  90 
S(DX/(P-R)  ) 


q 

S( 
X. 

rx 

T* 
R  ) 
.+- 
{=> 

+  1 
+•■? 
i  - 

S2 


OR/ (R*(C+4)*( l.-(R+OR)/AN) ) ) 
GT.06LTR  )  GO  TO  51 

R*(OA  )*( l.-R/AN) 

#0T 

(OA  )*0T  )  )/{  l.+R*(C  +  A)*DT/AN) 

I-X  )**2*Cl-CA*A**?*R)*0ELr 

t  IR  )+(SR(MS+l.  IR+D-SflCNS  +  1*  IR)  )*06I.R/0R 

♦  IR  )  +  (SR(NS  +  2»IR  +  l  )-SB(NS-*-2.TR)  )*OSLR/DR 

R2  )*(06LT-0T )/0T 


US 

N6N0.LT. '+5  )    GO    TO    8i 

=0 

100 

10UNS.rfX,R,XX,RR,P,^A,PR6S 

■t.INENG+1 

IR  )-PR6S 

LE. I  )    GO    TO    43 

42 


983888 

06 

LNS=1, NT 

l.R=l  ,NR 

»LR  )  =  SP(LNS,LR ) 


52 


r=i 

NS  =  NT 

OR-ES  =  -983888 
.  10  CONTINUE 
STOP 
END 
SENTRY 
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ABSTRACT 

The  importance  of  quantitative  techniques  in  decision  making 
emphasizes  the  need  of  efficient  techniques  as  a  tool  for  solving  manage- 
ment problems.  The  most  difficult  are  the  boundary  value  problems  with 
nonlinear  differential  and/or  difference  equations.  The  non-linearity 
in  the  system  equations  does  not  allow  the  application  of  superposition 
principle. 

One  of  the  most  important  techniques  developed  in  recent  years 
for  the  solution  of  optimization  problems  is  R.  Bellman's  dynamic  program- 
ming.  This  technique  solves  at  least  in  principle  a  large  number  of 
important  optimization  problems.   However,  because  of  extremely  large 
amount  of  fast  storage  memory  requirement,  called  by  Bellman  "curse  of 
dimensionality",  only  relatively  simple  problems  can  be  solved  on  exist- 
ing computers. 

State  increment  dynamic  programming,  developed  in  1965  by  R.  E. 
Larson,  requires  considerably  less  fast-access  memory  but  it  still  retains 
the  general  applicability  and  other  desirable  features  of  the  standard 
algorithm. 

In  this  thesis  first  a  brief  introduction  and  computational  pro- 
cedure of  state  increment  dynamic  programming  are  given.  Then  its 
application  to  advertisement  production  problem  with  two  state  variables 
and  a  control  variable  is  duscussed  in  detail. 

The  production  planning  with  consideration  of  advertisement  provides 
a  good  base  of  comparison  of  this  method  with  others  used  to  solve  the 
problems  with  same  model  in  the  past.   The  advantages  and  disadvantages 
of  state  increment  dynamic  programming  have  been  highlighted. 


